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ABSTRACT 



The undergraduate research-development program aims 
to increase students' orientation and commitment toward future 
research- dev elopment training and professional engagement. This paper 
focuses upon the instrument development training and professional 
engagement. This paper focuses upon the instrument development 
efforts in this area. Two new instruments were constructed: the 
Research Orientation Index (ROL) and an Omnibus Questionnaire to be 
used on a pretest-posttest basis. The ROI was constructed according 
to Likert's method of summated ratings; subjects were asked to place 
themselves on a 5-point continuum, ranging from strongly agree to 
strongly disagree. A pretest draft of a 82-item scale was 
administered to 204 selected education students. Their responses were 
submitted to a preliminary analysis, to a correlational analysis, and 
to a second analysis on the basis of which 25 items with the highest 
discriminations were selected. The Omnibus Questionnaire attempted to 
obtain information on biographical data; attitudes about existing 
undergraduate education programs at the Ohio State University, and 
attitudes toward research in terms of educational and occupational 
aspirations. (AF) 
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THE DEVELOPMENT AND EVALUATION OF UNDERGRADUATE 
AND GRADUATE RESEARCH CURRICULA 

Section III, Part Ila? Instrument Development 

Corann Okorodudu 
Glassboro State College 

American Educational Research Association 
Convention 
March, 1970 

The instruments and measures used in the first year of the undergraduate 
research-development program grew out of the program objectives and evaluation 
design alreacty discussed. As you may recall, the specific objectives of ' 
the program were: (l) to provide students with a basic knowledge of the 

methods of scientific inquiry j (2) to foster their development of skill 
in the application of such methods to educational problems, and (3) to in- 
crease their orientation and commitment toward fixture research-development 
training and professional engagement. The measurement of the knowledge and 
application objectives employed the usual criteria of student scores above 
X on non-standardized achievement tests and teacher judgment of student 
progress. My paper focuses upon the instrument development efforts during 
the summer of 1967 in relation to the measurement of the orientation, objec- 
tive. At that time, we were unable to find any ready-made instruments 
which could be used to measure the variables which we had identified pre- 
viously as relevant indices of research-development potential (Eargar, 
Okorodudu, Dworkin et al., 1967 ). Hence our decision to construct two new 
instruments — the Research Orientation Index (ROI) and an Omnibus Question- 
naire — and to use these on a pretest-posttest basis. 
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The Research Orientation Index (ROI) 

The Research Orientation Index is basically a measure of attitudes toward 
research. The theoretical position regarding the nature and structure of 
attitudes is derived from Katz and Stotland (1959)* An attitude is defined 
as an individual’s tendency or predisposition to evaluate an object or the 
symbol of an object by attributing qualities to it which can be placed along 
a dimension of desirability-undesirability. The basic structure of attitude 
thus is suggested to be determined largely by the affective or evaluative 
component, although the behavioral component and at least a minimal cognitive 
component increases the accuracy of prediction of behavior from the expression 
of attitude. Specified in terms of research, the affective component refers 
to positive or negative feelings associated with research, which feelings are 
subject to arousal by the presentation of statements or situations related to 
research. The behavioral component contains positive or negative action 
tendencies toward research. The cognitive component conta-ns beliefs about 
research, such beliefs tending in general to reinforce one's attitudes 
concerning research. 

The Research Orientation Index was constructed according to Likert's 
method of summated ratings (1932). According to this procedure, subjects 
are asked to place themselves on a five-point continuum, ranging from 
strongly agree to agree, uncertain, disagree, and strongly disagree. The 
five points are weighted 5-U-3-2-lin the case of favorable state- 
ments and 1-, 2-, U-, £>-, in the case of unfavorable statements. 

Subjects respond to each item on the basis of the extent to which they are 
willing tc endorse the item. Systematic variation in the subjects' reaction 
to the items of the scale are attributed to individual differences in the 
subjects, rather than differences in bhe items in terms of designated attributes. 
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The first step in the construction of the scale consisted of the 
collection of a series of 116 attitude statements from the literature on 
research or scientific inquirer in general and from individuals asked to 
express their opinions concerning research and research-related activities. 

As a second step, these statements were edited and checked for content 
validity by project staff according to standard procedures (Edwards, 1957). 
Finally, the statements were checked independently by three members of the 
staff to determine inter- judge agreement on whether each item expressed a 
clear-cut favorable or unfavorable attitude toward research. Items which 
showed disagreement between the judges were deleted. Eighty-two items, 
half favorable and half unfavorable were retained finally by this procedure. 
The third step consisted of a randomized ordering and printing of the 82- 
item scale, each item supplied with numbers corresponding to the five 
possible responses. 

During Speteniber and October 1967, the pretest draft of the scale was 
administered to a total of 20U selected education students at the Ohio State 
University, from a broad range of areas generally representative of potential 
research and non-research orientations. 

The directions were as follows: "The following are statements concern- 

ing attitudes toward several issues. Please indicate your agreement or 
disagreement with each statement by checking the answer that best describes 
how you personally feel, regardless of whether other people may agree or 
disagree with you. There are no right or wrong answers . The best answer 
is your honest, frank opinion. Do not look back and forth through the 
statements or try to remember how you checked similar items. Make a separate 
and independent judgment for each statement. Do not linger over individual 
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statements. There is no time limit, but respond as quickly as you can, and 
do not leave out any of the statements." Although the respondents were 
given as much time as neeued within the hour to complete the scale, the 
testing time was 2 0-2 5 minutes on the average. Each respondents record 
was scored to determine the degree of endorsement of each statement* A high 
score was considered to be an indication of a favorable attitude, while a 

o 

low score was taken as an indication of an unfavorable attitude. 

In order to select items for the shortened form of the Research Scale, 
statistical analyses were undertaken to determine (l) the reliability or 
internal consistency of the scale, and (2) the discrimination of the in- 
dividual statements. Plans were also made at that time for subsequent 
factor analyses, and tests of construct and concurrent validity. 

Data from 20U respondents were submitted to a preliminary analysis. 

The total number of scorable items was 82, with a score of 1 - 5 obtainable 
on each item. Thus the over-all range of obtainable scores was 82 - UlO. 

The scale mean and standard deviation actually obtained were 219.5 and 
22.9 respectively. The reliability of the scale in terms of internal 
consistency was determined by Kuder-Richardson Formulas No. 3 and No. lU, 
each of which yielded a coefficient of *91. Correlational analysis of 
items was performed to obtain information concerning the merit of each 
Item of the scale in relation to the total test score. This method of item 
analysis has been found to be quite comparable to t-tests of difference 
between a high and low group on Likert-type scales, (Edwards, 1957). A 
common procedure used in these analyses involves computing a Pearson r 
for each item with the total score minus the score for that item.. The 
range of coefficients was -.02 to .65. Both the range of coefficients and 
the scale standard deviation (22.9) indicated considerable variability in 
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how well each individual item is a measure of what the total item pool 
measures . 

A second analysis was carried out, using 58 items with correlation 
coefficients of .25 and higher with the total score for the initial pool of 
82 items. The second analysis involved data from all 20U respondents on the 
58 items selected on the basis of the preliminary analysis, With a score of 
1-5 obtainable, the over-sill possible range of scores was 58 - 290 . The 
scale mean and standard deviation actually obtained were 158.2 and 19.7 
respectively. Kuder Richardson Formulas No. 3 and lii and the split-half 
method each yielded a reliability coefficient of .92, which indicates some- 
what higher internal consistency among the 58 items. Again, a Pearson r 
for each item with the total score minus that item was computed for all 
58 items. The range of coefficients was .20 to .65. On the basis of this 
second analysis, 25 items with the highest discrimination (coefficients .U5 
and higher) were selected for the first shortened form of the Research 
Orientation Index, to be used in the evaluation of the first year of the 
undergraduate program. Later in September 1968 , a second form of the 
scale composed of 60 items, with coefficients of .30 or more, was construc- 
ted for the evaluation of the programs in the second year at the undergraduate 
and graduate levels. 

The Omnibu s Q u estionnaire 

The Omnibus Questionnaire was developed to obtain the following informa- 
tion: (l) biographical data} (2) attitudes about the existing undergraduate 

education program at the Ohio State University} and (3) attitudes toward 
research in terms of educational and occupational aspirations. In general, 
this instrument reflected a variety of factors which separate descriptive 
studies found to be strongly related to research involvement and productivity. 
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In terms of biographical data, the items included measures of socio- 
economic factors and details of personal educational history. Attitudes 
toward the undergraduate program attempted to assess student perceptions 
and evaluation of (l) the appropriates s of courcework in education in 
terms of academic, intellectual, and professional interests j (2) degree 
of freedom for self-direction; (3) faculty-strident interaction outside the 
context of the classroom and ( b ) the proportion of courses considered 
superior. Items relating to educational aspirations tapped the following 
aspects: (l) major area of specialization in graduate or professional 
school j (2) delayed versus immediate entry to graduate or professional 
school j (3) full-time versus part-time study in graduate or professional 
school; and (U) highest level degree realistically sought. In terms of 
occupational, aspirations, the items included participation in primarily 
research versus non-research roles in education as well as participation 
in specific research-development activities. 

Many of the items of the questionnaire pertaining to undergraduate 
education, educational and occupational aspirations, were constructed as 
Likert- type scales. Moreover, within each of the foregoing areas, items 
were weighted and combined to form indices. 
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